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Subject: Aviation Security: TSA's Change to Its Prohibited Items List Has Not Resulted 
in Any Reported Security Incidents, but the Impact of the Change on Screening 
Operations Is Inconclusive 

The alleged August 2006 terrorist plot to detonate liquid explosives onboard multiple 
commercial aircraft bound for the United States from the United Kingdom has 
highlighted both the continued importance of securing the civil aviation system and the 
potential that improvised explosive devices (IED) may be smuggled onboard passenger 
aircraft. The Transportation Security Administration (TSA) has primary responsibility for 
ensuring the security of civil aviation, which includes the safety of passengers and flight 
crew. 1 

One measure TSA uses to protect the aviation system is prohibiting individuals from 
carrying items that it determines to be a threat to the aircraft and its passengers into an 
airport sterile area or onboard an aircraft either in their carry-on bag or on their person. 2 
To implement this measure, TSA maintains a prohibited items list that informs both the 
Transportation Security Officers (TSO) who conduct passenger screening and the 



'See 49 U.S.C. §§ 114(d), 44903(b). 

2 

Sterile areas are located within the terminal where passengers are provided access to boarding aircraft. 
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traveling public of items that will not be allowed into an airport sterile area or onboard 
an aircraft. In December 2005, TSA revised its prohibited items list to allow passengers 
to carry: (1) metal scissors with pointed tips and a blade 4 inches or less in length as 
measured from the fulcrum; and (2) tools — such as pliers, screwdrivers, and wrenches — 
7 inches or less in length (excluding crowbars, drills, hammers, and saws)." 3 

TSA considers any incident that threatens the security or safety of an aircraft or its 
passengers and flight crew to be a security incident. These could include a range of 
activities onboard an aircraft such as disruptive passenger behavior, violence against a 
passenger or crew member, hijacking attempts, or the use of an improvised explosive 
device. By examining the security impacts of the change to the prohibited items list, this 
report considers the impacts that could result from a passenger attempting to use 
scissors or tools to hijack an aircraft or to commit other forms of violence onboard a 
flight. Such actions fall within TSAs statutory responsibility to ensure the safety and 
security of passengers and crew aboard aircraft. In accordance with Conference Report 
109-699, which accompanied the fiscal year 2007 Department of Homeland Security 
(DHS) appropriations act, 4 this report addresses the following questions: (1) What was 
TSAs basis for removing certain scissors and tools from the prohibited items list and 
what are stakeholder views on the change? (2) What have been the impacts, if any, of the 
removal of certain scissors and tools from the prohibited items list on the security of 
aircraft passengers and flight crew and on the effectiveness of checkpoint screening 
operations? 

To address these objectives, we analyzed TSA documentation and data, including TSA 
security incident reports, TSA written analyses related to the prohibited items list 
change, results of Threat Image Projection (TIP) testing, and data on training hours 
completed by TSOs. Although the TIP data we received had limitations, we believe that 
they are sufficiently reliable for the purposes of this report and that the data on training 
hours are sufficiently reliable as well. We also met with two Federal Security Directors 
(FSD) to obtain their views on the impact of the prohibited items list change on 
checkpoint screening operations. 6 However, information obtained from our interviews 
with these FSDs cannot be generalized because we did not use random selection or 
representative sampling when determining which FSDs should be interviewed. In 
addition, we met with officials at the Federal Aviation Administration (FAA) and the 
Federal Air Marshals Service (FAMS) — a component of TSA — to obtain their views 
regarding the prohibited items list change. We also met with TSA officials to obtain 
information on their rationale behind the change. We spoke with 13 stakeholders within 
the aviation industry, including representatives of 4 domestic aviation associations, the 
largest association representing airline pilots in the United States, the largest association 
representing flight attendants in the United States, an association representing federal air 



3 70 Fed. Reg. 72,930 (Dec. 8, 2005). 

4 

See H.R. Conf. Rep. No. 109-699, at 139 (2006) (accompanying H.R. 5441, enacted into law as the Department of 
Homeland Security Appropriations Act, 2007, Pub. L. No. 109-295, 121 Stat 1355 (2006)). 

5 

The Threat Image Projection system is designed to test TSOs' detection capabilities by projecting threat images, 
including guns, knives, and explosives, onto carry-on bags as they are screened during actual operations. TSOs are 
responsible for identifying the threat image and calling for the bag to be searched. Once prompted, TIP identifies to the 
TSO whether the threat is real and then records the TSO's performance in a database that could be analyzed for 
performance trends. 

6 FSDs are the ranking authorities responsible for leading and coordinating security activities at U.S. commercial 
airports at which TSA provides for or oversees the provision of screening activities. 
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marshals and other federal law enforcement officers, an international aviation 
association, and 5 aviation security experts. We also met with a major aircraft 
manufacturer to determine whether there are any major safety concerns related to the 
change to the prohibited items list. Finally, we incorporated aspects of a recently issued 
GAO report on passenger checkpoint screening procedures, which included a review of 
the factors TSA considered in modifying the prohibited items list and TSA's analysis 
supporting the December 2005 prohibited items list change. 7 We conducted our work 
from November 2006 through March 2007 in accordance with generally accepted 
government auditing standards. More details about the scope and methodology of our 
work are presented in enclosure I. 

Results in Brief 

TSAs stated purpose in removing certain scissors and tools from the prohibited items 
list was to shift TSO focus from items considered by TSA to pose a low threat (including 
certain scissors and tools) to items considered to pose a high threat, such as explosives. 
The change also was intended to better allocate TSA resources to implement other 
security measures that target explosives — a change supported by the majority of aviation 
industry stakeholders that we interviewed. TSA's decision to remove these items from 
the prohibited items list was based on the professional judgment of TSA officials that 
these items do not pose a significant threat to the security of the cockpit or to 
passengers and flight crew as well as internal studies that sought to examine, among 
other things, risks to flight security and considerations of customer concerns and 
screening efficiencies. As part of these internal studies, TSA collected data on the 
number and types of prohibited items surrendered at checkpoints and the time it takes 
for TSOs to conduct carry-on bag searches. In March 2007, we reported that TSA did not 
analyze these data to determine the extent to which TSO resources would actually be 
freed up to implement other security measures, nor did TSA analyze other relevant 
factors such as the amount of time taken to search for small scissors and tools and the 
number of TSOs conducting these searches. We recommended that TSA develop sound 
evaluation methods, when possible, that can help TSA determine whether proposed 
procedures would achieve their intended purpose. 8 TSA concurred with the 
recommendation and stated that it plans to make better use of generally accepted 
research design principles and techniques when operationally testing proposed changes 
to screening procedures. Based on our analysis of TSA data for the third and fourth 
quarters of fiscal year 2005 (a 6-month period), we determined that TSOs spent, on 
average, less than 1 percent of their time — about 1 minute per day over the 6-month 
period — searching for the approximately 1.8 million sharp objects, other than knives and 
box cutters, that were found at passenger screening checkpoints between April 2005 and 
September 2005. Therefore, it may not have been accurate for TSA to assume that no 
longer requiring TSOs to search for small scissors and tools would significantly 
contribute to TSAs efforts to free up TSO resources that could be used to implement 
other security measures. TSA acknowledged that its data collection and analysis effort 



GAO, Aviation Security: Risk, Experience, and Customer Concerns Drive Changes to Airline Passenger Screening 
Procedures, but Evaluation and Documentation of Proposed Changes Could Be Improved, GAO-07-57SU 
(Washington, D.C.: Mar. 7, 2007). The information in this report is considered sensitive security information in 
accordance with 49 C.F.R. part 1520 and is not available to the public. A public version of this report (GAO-07-634) is 
expected to be issued in May 2007. 

8 Ibid. 
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may not have been methodologically rigorous but stated that it did serve to provide 
insights regarding the type and quantity of items collected at the passenger checkpoint. 
TSA officials also stated that even if TSO resources were not freed up as intended, they 
continue to view their decision to allow small scissors and tools onboard aircraft as 
sound, particularly because their review of threat information determined that small 
scissors and tools do not pose a significant threat to aviation security. Additionally, 9 of 
the 13 aviation industry stakeholders whom we interviewed supported the removal of 
small scissors and tools from the prohibited items list because they believe small 
scissors and tools do not pose a risk to the security of the aircraft and stated that the 
change will increase TSAs focus on IEDs; the remainder disagreed, citing potential 
increased security risks. TSA officials acknowledged that small scissors and tools, as 
well as other items permitted onboard commercial aircraft, may potentially be used as 
weapons against passengers and flight crew, but stated that these items cannot be used 
to hijack an aircraft given the other layers of security in place, such as hardened cockpit 
doors that prevent unauthorized access to the flight deck. 

Based on our review of TSA security incident reports from the time period following the 
prohibited items list change (December 2005 through February 2007), there have been no 
reported security incidents onboard an aircraft involving the use of small scissors or 
tools. However, the impact of the prohibited items list change on security is uncertain 
because the absence of an event occurring involving the use of these items does not 
preclude the possibility that future occurrences could happen. In addition, with respect 
to the effectiveness of the change on checkpoint screening operations, it is not possible 
to determine this because the available data are inconclusive. As we reported in March 
2007, TSA conducted informal studies 30, 60, and 90 days following the change and 
concluded that TSO time was freed up to focus on high-threat items, but our analysis of 
TSA data does not support this conclusion. 9 TSA agrees that the agency could have 
conducted a more methodologically sound evaluation of the impact of the prohibited 
items list change, but continues to believe that the change nevertheless significantly 
contributed to the agency's efforts to free up TSO resources to focus on detecting high- 
threat items, such as explosives. It also is not clear whether the change had any impact 
on TSOs' ability to detect explosives — a key goal of the change. One way TSA measures 
the effectiveness of the passenger screening system in detecting threat items, such as 
explosives, is the results of threat image projection testing. 10 However, TSA does not 
claim nor do the data definitively support that TSAs change to the prohibited items list 
had any impact on threat image projection results because TSA implemented other 
changes to checkpoint screening operations at or around the same time as the prohibited 
items list change. With regard to TSAs efforts to increase training for identifying 
explosives as part of its overall effort to become a more risk-based organization, TSA 
data between October 2004 and January 2007 show an increase in the average number of 
hours spent in training per TSO, but this trend began before the change to the prohibited 
items list and there are other factors that may have contributed to this increase. 



Ibid. 

10 The results of TSA TIP testing are considered sensitive security information and thus could not be included in this 
report. 
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Background 



In accordance with applicable laws and regulations, TSA prohibits weapons, explosives 
or incendiaries, and other items that TSA believes pose a significant threat to civil 
aviation security onboard commercial aircraft. 11 TSA has divided these prohibited types 
of items into seven categories. Individuals are prohibited from carrying these items into 
an airport sterile area or onboard an aircraft either in their carry-on bag or on their 
person. Table 1 provides a description of the items included in the seven categories. 



Table 1 : Categories and Descriptions of Prohibited Items 

Category of prohibited item Description of items included in the category 

Guns and firearms BB guns; compressed air guns; firearms; flare pistols; gun lighters; parts of 

guns and firearms; pellet guns; realistic replicas of firearms; spear guns; 

starter pistols; stun guns/cattle prods/shocking devices. 

Sharp objects Axes and hatchets; bows and arrows; ice axes/ice picks; knives of any 

length, except rounded-blade butter and plastic cutlery; meat cleavers; 
razor-type blades, such as box cutters, utility knives, and razor blades not in 
a cartridge, but excluding safety razors; sabers; scissors, metal with pointed 
tips and a blade length greater than 4 inches as measured from the fulcrum; 

swords; throwing stars (martial arts). 



Club-like items 


Baseball bats; billy clubs; blackjacks; brass knuckles; cricket bats; golf 
clubs; hockey sticks; lacrosse sticks; martial arts weapons, including 
nunchucks, and kubatons; night sticks; pool cues; ski poles. 


All explosives 


Ammunition; blasting caps; dynamite; fireworks; flares in any form; 




gunpowder; hand grenades; plastic explosives; realistic replicas of 




explosives. 


Incendiaries 


Aerosol, any, except for personal care or toiletries in limited quantities; 
fuels, including cooking fuels and any flammable liquid fuel; gasoline; gas 
torches, including microtorches and torch lighters; lighter fluid; strike- 
anywhere matches; turpentine and paint thinner; realistic replicas of 
incendiaries; all lighters. 



Disabling chemicals and other Chlorine for pools and spas; compressed gas cylinders (including fire 
dangerous items extinguishers); liquid bleach; mace; pepper spray; spillable batteries, except 
those in wheelchairs; spray paint; tear gas. 



Tools Crowbars; drills and drill bits, including cordless portable power drills; 

hammers; saws and saw blades, including cordless portable power saws; 

other tools greater than 7 inches in length, including pliers, screwdrivers, 

and wrenches. 

Source: TSA. 



Passenger screening is a process by which personnel authorized by TSA inspect 
individuals and property to deter and prevent the carriage of any unauthorized explosive, 
incendiary, weapon, or other items included on TSA's prohibited items list onboard an 
aircraft or into a sterile area. 12 Passenger screening personnel — TSOs — must inspect 
individuals for prohibited items at designated screening locations. 13 As shown in figure 1, 
the passenger screening functions are 



See 49 U.S.C. § 44902; 49 C.F.R. §§ 1540.111, 1544.201(d). 

12 

Access to sterile areas is controlled by TSOs (or by nonfederal screeners at airports participating in the Screener 
Partnership Program) at checkpoints where they conduct physical screening of individuals and their carry-on baggage 
for weapons, explosives, and other prohibited items. 

13 

TSOs must deny passage beyond the screening location to any individual or property that has not been screened or 
inspected in accordance with passenger screening standard operating procedures. If an individual refuses to permit 
inspection of any item, that item must not be allowed into the sterile area or onboard an aircraft. 
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• X-ray screening of property, 

• walk-through metal detector screening of individuals, 

• hand-wand or pat-down screening of individuals, 

• physical search of property and trace detection for explosives, and 

• behavioral observation. 



Figure 1: Passenger Checkpoint Screening Functions 




Passenger screening functions 



Source: GAO and Nova Development Corporation. 

Notes: Explosive trace detection (ETD) works by detecting vapors and residues of explosives. Human operators collect samples by 
rubbing swabs along the interior and exterior of an object that TSOs determine to be suspicious, and place the swabs in the ETD 
machine, which then chemically analyzes the swab to identify any traces of explosive materials. 

Bomb Appraisal Officers (BAO) are available to respond to unresolved alarms at the checkpoint that involve possible explosive 
devices. The BAO may contact appropriate law enforcement or bomb squad officials if review indicates possible or imminent 
danger, in which case the BAO ensures that the security checkpoint is cleared. The BAO approves reopening of security lane(s) if 
no threat is posed. 

a BDOs are TSOs specially trained to detect suspicious behavior in individuals approaching the checkpoint. Should the BDO observe 
such behavior, he or she may refer the individual for individual screening or to a law enforcement officer. As of April 2007, BDOs are 
not present at all airport checkpoints. 

"The hand-wand or pat-down is conducted if a passenger is identified or randomly selected for additional screening because he or 
she met certain criteria or alarmed the walk-through metal detector. 

"Manual or ETD searches of accessible property occur if the passenger is identified or randomly selected for additional screening or 
if the screener identified a potential prohibited item on X-ray. 



Typically, passengers are only subjected to X-ray screening of their carry-on items and 
screening by the walk-through metal detector. Passengers whose carry-on baggage 
alarms the X-ray machine, who alarm the walk-through metal detector, or who are 
designated as selectees — that is, passengers selected by the Computer-Assisted 
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Passenger Prescreening System (CAPPS) or other TSA-approved processes to receive 
additional screening — are screened by hand-wand or pat-down and have their carry-on 
items screened for explosives traces or physically searched. 

In addition to passenger checkpoint screening, other layers of aviation security 
recognized by TSA include, among other things: 

• Hardened cockpit doors to prevent unauthorized access or forced entry to the 
flight deck. 

• Deployment of federal air marshals on certain flights to provide physical security 
should an incident occur. 

• Federal Flight Deck Officers (FFDO) Program to train pilots on commercial 
passenger and cargo aircraft on how to use lethal force against an intruder on the 
flight deck. 15 

• Security training for flight and cabin crews to handle potential threats onboard 
aircraft. Flight and cabin crews are expected to defend the flight deck in 
accordance with a TSA and FAA-developed guidance manual known as the 

Common Strategy. 16 

In addition, TSA considers the vigilance of able-bodied passengers to be an important 
layer of aviation security. Able-bodied passengers are those passengers who may engage 
in self-defense actions should an incident occur onboard commercial aircraft. 



CAPPS is a computer-assisted system that, based on information obtained from airline reservation systems, identifies 
passengers that may pose a high risk to aviation security. These high-risk passengers, along with other individuals 
selected for secondary screening, and their carry-on baggage are subject to additional and more thorough screening. 

15 

Administered by TSA, the FFDO Program deputizes volunteer pilots of commercial passenger aircraft as armed 
federal law enforcement officers for the purpose of defending the flight deck "against acts of criminal violence or air 
piracy." Since the program was officially established on February 25, 2003, TSA has deputized thousands of eligible 
flight crew members as FFDOs. 

16 The Common Strategy is a detailed guidance manual developed by TSA and FAA for pilots and other crewmembers 
to identify their responsibilities and the appropriate responses during in-flight security threats. In January 2005, TSA 
and FAA issued a revised version of the Common Strategy. The previous version, referred to as Common Strategy #1, 
was the strategy in effect on September 11, 2001. Common Strategy #1 was developed jointly by industry, FAA, and 
FBI, and presumed a hijacker whose motive might be ransom, escape from the law, political asylum, or publicity. 
According to the Common Strategy, the terrorist attacks of September 11, 2001, demonstrated that Common Strategy 
#1 was not effective in dealing with a new breed of hijacker whose motives are terrorism, mass murder, and suicide. 
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TSA Changed the Prohibited Items List to Shift TSO Resources to Higher- 
Threat Priorities and Most Aviation Industry Stakeholders Interviewed 
Supported TSA's Change 

TSA Conducted Various Studies to Determine Whether Changing the Prohibited Items 
List Would Free Up TSO Resources, but Some Efforts Lacked Methodological Rigor 



As we reported in March 2007, TSA changed the prohibited items list in an effort to shift 
TSO resources to focus on higher threats, such as explosives, and based on its 
determination that small scissors and tools do not pose a risk to aviation security. 18 TSA's 
decision was informed by the conclusions reached by an Explosives Detection 
Improvement Task Force established in October 2005 by the TSA Assistant Secretary to 
respond to the threat of IEDs being carried through the checkpoint. The goal of the task 
force was to apply a risk-based approach to screening passengers and their baggage in 
order to enhance TSA's ability to detect IEDs. As part of its analysis, the task force 
considered a number of factors including threat information, TSO effectiveness, and 
overall screening performance. According to TSA officials, the task force also considered 
the results of a Prohibited Items Working Group that was established in February 2005 
by the then-TSA Assistant Secretary to develop recommendations for modifying the 
prohibited items list to better reflect the current aviation security environment. 

The Prohibited Items Working Group assessed each item on the prohibited items list 
using four criteria: (1) risks to flight security (i.e., can the item be used to take down an 
aircraft in flight); (2) legal restrictions (i.e., hazardous and other materials that are 
prohibited from the aircraft or from the flight cabin); (3) public concern and screener 
effectiveness (i.e., would permitting the item onboard an aircraft cause significant 
passenger and flight crew concern regarding their safety); and (4) international 
standards (i.e., international protocols recommend that the item be prohibited from the 
aircraft or the flight cabin). At the conclusion of its analysis, the working group 
recommended that scissors with pointed tips less than 6 centimeters (2.36 inches) and 
tools less than 7 inches be removed from the prohibited items list because these items 
were not considered to represent a risk to the aircraft or cockpit security. Although the 
working group based its size restriction for scissors on the size parameters 
recommended by the International Civil Aviation Organization (ICAO) — which is to 
provide for the safe, orderly, and efficient development of international civil aviation — 
the working group deviated from ICAO's recommendation to prohibit all pointed/edged 



17 GAO-07-57SU. 

18 The change to the prohibited items list was one of several other changes to TSA procedures intended to focus more 
TSA resources on higher threats, such as explosives. The Explosive Detection Improvement Task Force recommended 
seven proposed procedures that were ultimately implemented by TSA. These procedures were considered by senior 
TSA officials as especially important for enhancing the detection of explosives and for deterring terrorists from 
attempting to carry out an attack. According to TSA, some of the proposed procedures, such as the prohibited items 
list change, could also free up TSOs so that they could spend more time on procedures for detecting explosives and 
less time on procedures associated with low security risks, such as identifying small scissors in carry-on bags. The 
seven proposed procedures tested by the task force reflect both new procedures and modifications to existing 
procedures. The procedures are discussed in detail in GAO-07-57SU. 
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scissors. A TSA representative from the working group stated that this change was 
recommended because the working group concluded that pointed/edged scissors could 
not be used to gain access to the cockpit to take down an aircraft in flight. In addition, 
the working group stated that concentrating on such items diminished TSA's efforts to 
focus on identifying objects that pose the greatest threat to aviation security, such as 
IEDs. 

Subsequent to the analysis of the working group, TSA's Explosives Detection 
Improvement Task Force collected information from several sources to test its 
assumption that a disproportionate amount of TSO resources was being spent searching 
for small scissors and tools. First, TSA reviewed data maintained in TSA's Performance 
Management Information System, 20 which showed that during the third and fourth 
quarters of fiscal year 2005 (a 6-month period), TSOs collected a total of about 1.8 million 
sharp objects other than knives or box cutters and about 468,000 tools. The sharp objects 
constituted 19 percent of all prohibited items surrendered at the checkpoint during this 
period and tools constituted 5 percent of the items. Second, based on information 
provided by FSDs, TSOs, and other screening experts, TSA determined that scissors 
constituted a large majority of the total number of sharp objects found at passenger 
screening checkpoints. TSA also concluded that small screwdrivers, wrenches, and pliers 
make up a large majority of the tools found at checkpoints. Third, TSA headquarters 
officials searched through surrendered items bins at four airports and found that most of 
the scissors had blades less than 4 inches in length and a very large percentage of the 
tools that were surrendered were 7 inches or smaller. 

Based on these collective efforts, TSA's Explosive Detection Improvement Task Force 
concluded that a significant number of items found at the checkpoint were low-threat, 
easily identified items, such as small scissors and tools, and that a disproportionate 
amount of time was spent searching for these items — time that could have been spent 
searching for high-threat items, such as explosives. The task force also concluded that 
because TSOs can generally identify scissors and tools on X-ray images easily, if small 
scissors and tools were no longer on the prohibited items list, TSOs could avoid 
conducting time-consuming physical bag searches to locate and remove these items. TSA 
ultimately concurred with the recommendations provided by the Explosive Detection 
Improvement Task Force and decided to remove scissors less than 4 inches and certain 
tools less than 7 inches from the prohibited items list. 

Although TSA's rationale for its December 2005 change to the prohibited items list was 
to reduce focus on low-threat items in order to free up TSO time, attention, and 
resources to implement screening practices that better focus on high-threat items — such 
as Screening Passengers by Observation Technique (SPOT) and Unpredictable Screening 



Nations that are members to ICAO agree to cooperate with other member states to meet standardized international 
aviation security measures. ICAO recommends that pointed/edged scissors of any size should be prohibited from the 
flight cabin, while rounded or blunt scissors less than 6 cm should be permitted in the flight cabin. The TSA Prohibited 
Items Working Group utilized the ICAO size parameters, but applied the parameters to both rounded/blunt scissors as 
well as pointed/edged scissors, thus deviating from the ICAO recommendation to ban all pointed/edged scissors. 

20 

TSA s Performance Management Information System is designed to collect, analyze, and report passenger and 
baggage screening performance data, such as wait times at selected airports, workload data, and the performance and 
utilization of passenger and baggage screening equipment. TSA headquarters uses the Performance Management 
Information System data to support external reporting on performance and internal decision-making processes. 
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Process (USP) — we reported in March 2007 that TSA had not conducted the necessary 
analysis of the data collected to determine the extent to which the removal of small 
scissors and tools from the prohibited items list could free up TSO resources. 
Specifically, we found that TSA had not analyzed the data on sharp objects surrendered 
at the checkpoint along with other relevant factors, such as the amount of time taken to 
search for scissors and the number of TSOs at the checkpoint conducting these searches. 
Based on our analysis of TSA's data for the third and fourth quarters of fiscal year 2005 
(a 6-month period), where we considered these other relevant factors, we determined 
that TSOs spent, on average, less than 1 percent of their time — about 1 minute per day 
over the 6-month period — searching for the approximately 1.8 million sharp objects, 
other than knives and box cutters, that were found at passenger screening checkpoints 
between April 2005 and September 2005. If the average amount of time TSOs spent 
searching for sharp objects per day over a 6-month period was less than 1 minute per 
TSO, and sharp objects constituted just 19 percent of all prohibited items surrendered at 
checkpoints over this period, then it may not be accurate to assume that no longer 
requiring TSOs to search for small scissors and tools would significantly contribute to 
TSA's efforts to free up TSO resources that could be used to implement other security 
measures. TSA stated that the decision to remove small scissors and small tools from the 
prohibited items list was not only based on an analysis of data but was also firmly rooted 
in its assessment of risk, professional judgment, and experience. According to TSA, this 
included interviews with FSDs who unanimously indicated the change would free up 
TSO resources, as well as examinations of the prohibited items surrendered at several 
airports and a study to determine the amount of time taken to conduct bag searches. 

TSA acknowledged that this particular data collection and analysis effort may not have 
been methodologically rigorous, but stated that it did serve to provide insights regarding 
the type and quantity of items collected at the passenger checkpoint and the analysis 
effort generally supported the decision. Additionally, the TSA Assistant Secretary stated 
that even if TSA determined that the proposed prohibited items list modification would 
not free up existing TSO resources to conduct explosives detection procedures, he 
would have implemented the change anyway considering that such items no longer 
posed a significant security risk given the multiple layers of aviation security. 

In our March 2007 report, we recommended that TSA develop sound evaluation methods, 
when possible, that can help TSA determine whether proposed procedures that are 
operationally tested would achieve their intended purpose, such as enhancing TSA's 
ability to detect prohibited items and freeing up existing TSO resources that could be 
used to implement proposed procedures. 22 TSA concurred with the recommendation and 
stated that it plans to make better use of generally accepted research design principles 
and techniques when operationally testing proposed changes to screening procedures. 
For example, TSA agreed to consider using random selection, representative sampling, 
and control groups in order to isolate the impact of proposed changes to screening 
procedures from the impact of other variables. 



Screening Passengers by Observation Technique involves specially trained TSOs observing the behavior of 
passengers and resolving any suspicious behavior through casual conversation with passengers and referring 
suspicious passengers to selectee screening. Unpredictable Screening Process entails random selection of passengers 
across two screening lanes to be subjected to a predetermined element of the selectee screening process. 

22 

GAO-07-57SU. 
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Most Aviation Industry Stakeholders We Contacted Supported TSA's Changes to the 



Prohibited Items List 



The majority (9 of 13) of the aviation industry stakeholders that we interviewed 
supported the removal of small scissors and tools from the prohibited items list. In 
general, these stakeholders said that they believe that the layers of aviation security 
reduce a passenger's ability to access the cockpit with low-threat items, and further 
noted that passengers may carry other items onboard an aircraft (such as glass bottles, 
pens, and sharpened credit cards) that may also be used as weapons. Stakeholders also 
stated that TSOs will be able to better focus on detecting IEDs if low-threat items such as 
small scissors and tools are removed from the prohibited items list. However, 4 out of 
13 aviation industry stakeholders that we interviewed were opposed to the prohibited 
items list change, stating that permitting scissors increases the risk of violence against 
passengers and flight crew onboard an aircraft. Some of these stakeholders also stated 
that scissors also increase the risk that hijackings could be successfully implemented 
because scissors have bladed edges and pointed tips and therefore can be used as knives, 
and because terrorists can train with scissors to perfect their use as weapons. These 
stakeholders further stated that unlike other items that can be improvised to create a 
cutting surface (such as broken glass bottles), terrorists would not need to alter scissors 
onboard aircraft to use them as weapons. This could also allow a passenger to use the 
cutting edge and/or the sharpened tip of a scissor as a weapon without alerting other 
passengers or flight crew, as compared with the attention that could be drawn to a 
passenger that breaks a glass bottle. 

TSA acknowledges that scissors and tools may be used as weapons against passengers 
and flight crew. However, TSA stated that other items that are permitted onboard 
commercial aircraft, such as pens and glass bottles, may also be used as weapons against 
passengers and flight crew. TSA also maintained that its focus is on detecting explosives 
or items that can be used to breach the cockpit and potentially hijack the aircraft, which 
TSA and the majority of the aviation industry stakeholders that we spoke with view as a 
significant threat to aviation. TSA maintained that small scissors and tools cannot be 
used to hijack an aircraft, particularly given the other layers of security. 

Although stakeholders who both supported and disagreed with TSA's change stated that 
the layers of security implemented since September 11, 2001 — particularly the hardened 
cockpit door — have decreased the likelihood of a successful hijacking, stakeholders 
generally stated that the risk of a hijacking is highest when the cockpit door is opened. In 
an attempt to mitigate this potential vulnerability, and in accordance with the air 
carrier's responsibility to ensure that no passenger can access the flight deck when the 
cockpit door needs to be opened during flight, air carriers will typically place a beverage 
cart between passengers and the cockpit with a flight attendant standing behind the cart. 
The beverage cart and the flight attendant serve as a "secondary barrier" between 
passengers and the cockpit door. However, two aviation stakeholders — a former law 
enforcement officer who provides self-defense training and a representative from the 
association of flight attendants — stated that this secondary barrier can be circumvented 
by a determined terrorist using a scissor to attack the flight attendant who is manning 
the beverage cart, which could allow the terrorist to negotiate around the beverage cart 
and then access the open cockpit door. A senior TSA official stated that flight crew 
protocols are sufficient to ensure passengers cannot breach the cockpit and that 
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mechanisms are in place to ensure that cockpit doors are opened for brief periods of 
time. 



No Security Incidents against Passengers or Crew Using Scissors or Tools Have 
Been Reported to TSA Since the Change to the Prohibited Items List, but the 
Impact of the Change on Screening Operations Is Inconclusive 

No Onboard Incidents Involving Small Scissors or Tools Reported to TSA Since 
Prohibited Items List Change and FAA Does Not Believe These Items Pose a Risk to the 
Integrity of an Aircraft 

Based on our review of TSA security incident reports from the time period following the 
prohibited items list change (December 2005 through February 2007), there have been no 
reported security incidents onboard an aircraft involving the use of small scissors or 
tools. 23 However, TSA and aviation security stakeholders we spoke with acknowledged 
that the absence of an onboard incident involving scissors or tools as weapons does not 
preclude the possibility of such an incident in the future. In addition, based on aircraft 
vulnerability and system safety and security analyses performed to date by government 
and industry, neither FAA nor a major aircraft manufacturer we interviewed perceive any 
meaningful increase in risk to the integrity of an aircraft associated with TSAs decision 
to permit small scissors and tools onboard aircraft. 24 FAA officials also stated that 
aircraft are designed so that there are many layers of protection to prevent damage to the 
integrity of an aircraft from within (e.g., hardened cockpit doors and separate and 
redundant wiring for critical systems with few internal access points). The aircraft 
manufacturer stated that while it is possible that terrorists or others intending to do 
harm to the aviation system could use these items in ways not currently foreseeable, 
given current risk mitigation activities, the ability to inflict major damage to an aircraft 
with them is extremely remote. 

Impact of Prohibited Items List Change on Checkpoint Screening Operations Is 
Inconclusive 

TSA conducted informal studies 30, 60, and 90 days after the prohibited items list 
changes went into effect to determine whether the change had resulted in reductions in 
the percentage of carry-on bags that were searched and overall screening time. However, 
in a prior report, we identified limitations in TSAs methodology for conducting these 
studies and concluded that it may not be accurate to assume that the prohibited items 



Because TSA is the primary agency responsible for aviation security and maintains records of aviation security 
incidents, TSA security incident reports were our primary source of information for identifying incidents involving 
small scissors or tools. These security incident reports summarize transportation security incidents — including 
aviation — that are reported to TSA and include descriptions of the incident. We used the December 2005-February 
2007 time period because it was after the effective date of the prohibited items list change. Pursuant to TSA-issued 
Security Directive 1544-04-15, all aircraft operators are required to immediately report all threats that could affect the 
security of commercial aircraft to TSA. 

24 The Federal Aviation Administration has primary responsibility for ensuring the safety of civil aviation operations, 
including the operation of air traffic control and regulating the manufacture, operation, and maintenance of aircraft. 
See 49 U.S.C. § 44701. 
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list change freed up resources. TSA agrees that the agency could have conducted a 
more methodologically sound evaluation of the impact of the prohibited items list 
change, but TSA continues to believe that the change did significantly contribute to the 
agency's efforts to free up TSO resources to focus on detection of high-threat items, such 
as explosives. TSA officials stated that that they have not conducted or planned any 
additional studies on the prohibited items list change to determine the impact of the 
change on the effectiveness of screening operations. Officials continue to view the 
change as sound based on their professional judgment and assessment of risk, and state 
that the change allowed the agency to shift focus from low risks to areas such as 
increased focus on explosive devices and increased training. 

In February 2007, a TSA official stated that some FSDs interviewed several TSOs after 
the prohibited items list change went into effect, and these TSOs reported that the 
change did save screening time. However, TSA could not identify how many TSOs were 
interviewed, at which airports the TSOs were located, and how the TSOs were selected 
for the interview; nor did TSA document the results of these interviews. As TSA did not 
use random selection or representative sampling when determining which TSOs should 
be interviewed, the interview results cannot be generalized. 

Most of the FSDs we interviewed in August 2006 as part of our passenger screening 
procedures review stated that the prohibited items list change, in addition to another 
change, did not collectively free up TSO resources to perform screening activities 
focused on threats considered to pose a high risk, such as explosives.* Specifically, 13 of 
19 FSDs we interviewed at airports that tested USP and SPOT said that TSO resources 
were not freed up as a result of the prohibited items list change and another change 
made by TSA during this time frame. 27 In addition, 9 of the 19 FSDs said that in order to 
operationally test the procedures, TSOs had to work overtime, switch from other 
functions (such as checked baggage screening), or a screening lane had to be closed. 
Moreover, 13 of the 19 FSDs stated that TSOs did not experience more time to conduct 
explosives training. 28 

In addition to the lack of clarity about the impact of changes to the prohibited items list 
on TSO's available time, it also is not clear whether the change had any impact on TSOs' 
ability to detect IEDs — a key goal of the change. The results of threat image projection 



The results of the informal follow-on studies, which were conducted at 6 to 9 airports, show that the percentage of 
carry-on bags searched increased by about 0. 1 percentage point at the time of the 30-day study, then decreased by 
2.3 and 0.7 percentage points at the time of the 60-day and 90-day studies, respectively. However, the results of these 
informal studies may not be reliable due to the limitations in the methodology TSA used to conduct the studies. 
Specifically, TSA did not use a methodology that would control for factors other than the prohibited items list change 
that may influence the percentage of carry-on bags searched by TSOs. To do this, TSA would have had to develop a 
formal, systematic methodology for randomly selecting various times of day, location of checkpoints, number of 
checkpoints, and so on for data collection. By not controlling for such factors, TSA may not know the extent to which 
a reduction in the percentage of carry-on bags searched is due to the prohibited items list changes. See GAO-07-57SU. 

26 

See GAO-07-57SU. An additional measure intended to free up TSO resources involved changes to CAPPS rules to 
reduce the number of passengers selected for secondary screening. TSA's assumption is that these changes could 
allow TSOs who were normally assigned to selectee screening duties to be reassigned to new procedures, which may 
require new screening positions. 

27 

Since we did not use random selection or representative sampling when determining which FSDs should be 
interviewed, the interview results cannot be generalized. 

28 Of the remaining 6 FSDs, 5 said that TSO resources were freed up as a result of the prohibited items list and CAPPS 
rules changes, and 1 was uncertain whether TSO resources were actually freed up. 
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(TIP) testing are one way that TSA measures the effectiveness of the passenger 
screening system in detection of threat items, such as explosives. The results of TSA TIP 
testing are considered sensitive security information and thus could not be included in 
this report. Nevertheless, it is not clear whether TSA's change to the prohibited items list 
had any impact on TSOs' ability to identify IEDs during TIP testing because multiple 
factors could have accounted for the changes in TIP scores over time. For example, TSA 
implemented other changes to checkpoint screening operations at or around the same 
time as the prohibited items list change. These changes include both new and revised 
procedures, such as: revising the USP to include selected screening process elements 
like explosive trace detection of footwear and accessible property; screening 100 percent 
of passengers' footwear; banning liquids and gels; revising bulk-item pat downs to 
include the waistline down to the ankles; targeting threat area searches within baggage; 
revising the CAPPS rules; and implementing the new SPOT procedure. In fact, FSDs we 
interviewed at two category X airports 29 in February 2007 as well as other TSA officials 
stated that at this time it is not possible to isolate the effect of the prohibited items list 
change from these additional changes in order to determine its impact on checkpoint 
screening operations and whether the prohibited items list change freed up TSO 
resources. 30 

With regard to TSA's efforts to increase training for identifying IEDs as part of its overall 
effort to become a more risk-based organization, TSA data between October 2004 and 
January 2007 show an increase in the average number of hours spent in training per TSO, 
but this trend began before the change to the prohibited items list and there are other 
factors that may have contributed to this increase. Our analysis of these data show an 
increase of an average of 1.68 hours per TSO in monthly IED training over the 29-month 
period, from an average of 0.42 hours per TSO in October 2004 to an average of 2.10 
hours per TSO in February 2007. According to TSA's TSO training officials, there are two 
primary explanations for the increase: (1) in October 2005 TSA provided a 4-hour IED 
training course to 18,000 TSOs over a 3-week period, and, according to TSA, about 98 
percent of the 48,236 TSOs on board had received classroom, checkpoint, or computer- 
based improvised explosive device recognition training as of February 6, 2007; and 
(2) in May 2006 TSA instituted a new monthly requirement of 4 hours of IED training per 
TSO. Thus, although a goal of the prohibited items list change was to increase TSO 
training hours for detecting IEDs, TSA program officials acknowledge, and we agree, that 
it is not clear whether the change to the prohibited items list had any impact on time 
spent in training. 

Concluding Observations 

TSA is faced with the challenge of addressing numerous threats to commercial aviation 
security, as demonstrated by the alleged August 2006 terrorist plot to detonate liquid 
explosives onboard multiple commercial aircraft bound for the United States from the 
United Kingdom. TSA's December 2005 change to the prohibited items list is one of 
several efforts TSA has made to focus its resources on addressing the threat posed by 
explosives, which TSA considers to be the most significant threat to commercial aviation 



TSA classifies the over 400 commercial airports in the United States into one of five categories — X, I, II, III, and IV. 
Generally, category X airports have the largest number of passenger boardings and category IV airports have the 
smallest number. 

30 

The views of the two FSDs we interviewed may not be representative of the views of the population of FSDs. 
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security. While TSA's consideration of threat information, the professional judgment of 
TSA personnel, data analysis, and international standards all constitute reasonable 
inputs to making informed decisions on how to best anticipate and address threats given 
its available resources, the impact of the prohibited items list change on security and 
screening effectiveness is inconclusive. Nevertheless, we are encouraged that TSA 
recognized the limitations in its analysis of data used to help inform the prohibited items 
list change and plans to improve the methodological rigor for evaluating proposed 
changes to passenger screening procedures in the future, as we recommended in our 
March 2007 report. This effort will be particularly important as additional changes to 
passenger screening procedures — including future revisions to the prohibited items 
list — are considered and implemented. 

Agency Comments 

We provided a draft of the report to DHS for its review and comment. TSA provided 
technical comments, which we incorporated as appropriate. 

We will send copies of this report to the Secretary of the Department of Homeland 
Security; the Assistant Secretary, TSA; and interested congressional committees as 
appropriate. We will also make this report available at no charge on GAO's Web site at 
http://www.gao.gov . If you or your staff have any questions about this report, please 
contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of 
Congressional Relations and Public Affairs may be found on the last page of this report. 
Key contributors to this report are listed in enclosure II. 




Robert Goldenkoff 

Acting Director, Homeland Security and Justice Issues 



Enclosures 
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Enclosure I: Scope and Methodology 



To describe TSA's basis for removing certain scissors and tools from the prohibited 
items list and stakeholder views on the change, we obtained and analyzed TSA 
documentation of the proposed prohibited items list change considered by TSA's 
Explosives Detection Improvement Task Force, which was the deliberating body for 
proposed TSA procedural changes that were considered between October 2005 and 
December 2005. We also obtained and analyzed a draft TSA Prohibited Items Working 
Group analysis, as well as TSA public statements and testimonies regarding the rationale 
for the prohibited items change. We also met with TSA officials to obtain information on 
their rationale behind the change. In addition, we met with officials at the Federal 
Aviation Administration (FAA) and the Federal Air Marshals Service (FAMS) — a 
component of TSA — to obtain their views regarding the prohibited items list change. We 
spoke with organizations within the aviation community including four domestic aviation 
associations, one international aviation association, a major aircraft manufacturer, the 
largest association representing airline pilots in the United States, the largest association 
representing flight attendants in the United States, and an association representing 
federal air marshals and other federal law enforcement officers. 31 In addition, we met 
with five aviation security experts to obtain their views on TSA's change to the 
prohibited items list. 32 We selected these experts based on their depth of experience in 
the field of aviation, employment history, and their recognition in the aviation security 
community. We also met with a major aircraft manufacturer to determine whether there 
are any major safety concerns related to the change to the prohibited items list. Finally, 
we incorporated aspects from our recently issued report on passenger checkpoint 
screening procedures, 33 which included a review of the factors TSA considered in 
modifying the prohibited items list and TSA's analysis supporting the December 2005 
prohibited items list change. 

To determine the impacts, if any, that the removal of certain scissors and tools from the 
prohibited items list had on security and on the effectiveness of screening operations, we 
obtained and reviewed TSA documentation and data including the results of threat image 
projection (TIP) testing and data on training hours completed by Transportation Security 
Officers (TSO). We sent written questions about data quality control and reporting 
procedures to TSA officials responsible for collecting and analyzing these data, and 
received responses to these questions. The TIP data TSA provided contained limitations. 
First, the data contained only monthly averages for tests in which improvised explosive 
devices (IED) images had been successfully identified by TSOs, according to individual 
airports in each airport category; we did not receive the raw numbers of image 
presentations from which the percentages were derived. Therefore, to compute an 
average percentage of successful TIP tests across all airports, we computed an average 
of averages. Computing an average in this manner can provide a result that is slightly 
different than if raw data had been used. For example, we could not adjust our 



Specifically, we met with the Air Transport Association (ATA), the National Air Carrier Association (NACA), the 
Regional Airline Association (RAA), the Air Carrier Association of America (ACAA), the International Air Transport 
Association (IATA), the Air Line Pilots Association (ALPA), the Association of Flight Attendants (AFA), and the 
Federal Law Enforcement Officers Association (FLEOA). 

32 

The views of these five experts may not necessarily represent the general view of other experts in the field of 
aviation security. 

GAO-07-57SU. 
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computations to account for differing numbers of image presentations or the rate of 
image presentations by airport. Second, there were missing values, or no test results, for 
some airports in certain months. Despite these limitations, we believe the TIP data were 
sufficiently reliable to provide an indication of TSOs' abilities to identify IED images. In 
addition, we interviewed Federal Security Directors (FSD) from Boston Logan Airport 
and Washington Dulles International Airport to obtain anecdotal information about their 
views on the impact of the prohibited items list change on checkpoint screening 
operations. However, the perspectives of these two FSDs cannot be considered to be 
representative of the views of FSDs nationwide or generalized because we did not use 
random selection or representative sampling when determining which FSDs should be 
interviewed. 

To determine whether the change to the list of prohibited items had any impact on TSO 
time spent in training, we also analyzed training data provided by TSA on the average 
number of hours spent in training per TSO for the period from October 2004 through 
February 2007. TSA uses a dynamic system to capture training data called the Online 
Learning Center, and TSA offers several reasons for the dynamic nature of this system. 
First, TSA employees and contractors are continuously allowed to update training 
history hours. As a result, data on training hours and attendance extracted from the 
database at two different points in time may vary as employees and contractors update 
their training history. Second, there can be a delay in updating training data due to 
manual entry of student results. TSA policy is that final reports are generated on the 10 th 
of each month in order to permit time to collect and consolidate airport data for manual 
data entry. The training hour data were sufficiently reliable for our purpose in showing a 
general increase in IED training over time. Our results are based on the data TSA 
provided to us on March 21, 2007. 

To determine if any security incidents onboard an aircraft involving the use of small 
scissors or tools were reported to TSA, we reviewed and analyzed TSA security incident 
reports from the time period following the prohibited items list change (December 22, 
2005 — the effective date of the prohibited items list change — through February 28, 2007). 
Because TSA is the agency with primary responsibility for aviation security and 
maintains records of aviation security incidents, TSA security incident reports were our 
primary source of information for identifying incidents involving small scissors or tools. 
We followed a two-step process to identify incidents appropriate to our review. During 
the first step, one analyst reviewed all incidents in each daily TSA security incident 
report to identify any incidents that he or she discerned involved small scissors or tools 
based on key words or phrases in the incident title or description. A log was created for 
each incident report reviewed. In the second step, a random sample of 10 percent of the 
incident reports was selected, and these reports and their accompanying logs were 
reviewed by a second analyst to verify the accuracy of the first analyst's judgments. We 
limited the scope of our TSA security incident report review to incidents that occurred 
on commercial passenger aircraft in-flight. We defined "in-flight" as the time between 
aircraft take-off and landing. Although it is possible that there were some incidents 
involving small scissors or tools that occurred during the time period of our review that 
were not reported to TSA, and thus not recorded in the incident reports, we found the 
incident reports sufficiently reliable for our purposes. 

We conducted our work from November 2006 through March 2007 in accordance with 
generally accepted government auditing standards. 
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Enclosure II 



GAP Contact and Staff Acknowledgments 

GAP Contact 



Robert Goldenkoff (202) 512-2757 or goldenkoffr@gao.gov . 
Staff Acknowledgments 

Key contributors to this report were Maria Strudwick, Assistant Director; 
David Alexander; Christopher Backley; Amy Bernstein; Tony Cheesebrough; 
Adam Hoffman; Stanley Kostyla; Tom Lombardi; and Brian Sklar. 
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GAO's Mission 



The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting its 
constitutional responsibilities and to help improve the performance and 
accountability of the federal government for the American people. GAO 
examines the use of public funds; evaluates federal programs and policies; 
and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO's 
commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 
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